Agent, method and computer system for negotiating in a virtual environment

ABSTRACT

A multi-agent system is described that uses software agents to represent objects in a chosen real world environment. Each agent has structure having certain characteristics where a relation with other agents can be established if the other agents have characteristics that are suitable to the requesting agent. The multi-agent system is able to perform dynamic negotiations autonomously between agents so as to establish relations when a new event enters the system or to offer compensation to modify existing relations and to self-organise to optimise the overall value of the system.

RELATED APPLICATIONS

This application is the U.S. National Stage of International ApplicationNo. PCT/GB03/00443, filed Feb. 3, 2003, published in English, and claimspriority under 35 U.S.C. § 119 or 365 to Great Britain Application No.0202527.8, filed Feb. 4, 2002.

The present invention is concerned with managing a virtual environmentand is particularly but not exclusively concerned with software agentsand a computer system for executing such agents. The invention is alsoconcerned with a method of simulating a real environment and of managingdemands and resources using a simulated environment.

Computer simulations of real environments have increasing importance inthe modern world. Many systems, traditionally placed under humancontrol, have recognised the benefits of automatic control performed bycomputers. This is evident in most sectors, for example an industrialproduction line, auto-pilot function on aircraft, etc. The considerableprocessing power and consistency of output afforded by modern computingplatforms, have resulted in the growing trend for placing systems undercomputer control. The level of autonomous behaviour, that is independentof human intervention, is governed only by the perceived intelligence ofthe system.

Computing concepts such as artificial intelligence, fuzzy logic, neuralnetworks, etc are often used to increase the so-called “intelligence”control of a system. In particular, the growth of so-called “agents” hasincreased. An agent, as described herein, is a software object capableof communicating autonomously and intelligently with other agents.

A system can be modelled by identifying the key elements of the systemand then developing specialised agents that have the same properties andattributes to represent those key elements.

The problem of the allocation of resources is well understood for thecase when all resources and all demands for resources in a system areknown in advance, and can be solved using existing technology. However,real market conditions are so turbulent that often the demand forresources is subject to frequent and unpredictable changes. Theallocation of resources is dynamic. Under such conditions algorithms forthe allocation of resources must include a provision for frequent andrapid re-allocation. Conventional programs cannot cope with theseconditions.

Multi-agent systems offer an effective method for the dynamic allocationand re-allocation of resources to frequently changing demands.

One object of the present invention is to provide a modelling technique,and components associate therewith capable of the dynamic allocation ofresources in a system.

According to a first aspect of the present invention there is providedan agent adapted for negotiating in a virtual environment, the agentcomprising: an agent descriptor implemented as an executable program andcomprising a set of properties which determine the negotiatingcharacteristics of the agent; and an agent body implemented as anexecutable program and comprising at least one sensor for receivinginformation from the virtual environment, at least one actuator forinteracting with the environment, and a decision engine operable in anactive state during a predetermined time period to execute a decisionmaking process based on the information received from the at least onesensor wherein the decision engine is operable to establish a relationwith another agent in dependence on the negotiating characteristics andto determine a value representing the quality of the establishedrelation.

According to a second aspect of the present invention there is provideda method of simulating a real environment by creating a virtualenvironment comprising a plurality of agents as described above, themethod comprising: for at least some of said agents, invoking an activestate of the agent for a predetermined time period; for each activeagent, implementing a decision making process based on the informationreceived from the at least one sensor according to the negotiatingcharacteristics of the agent to determine whether a relation can beestablished with another agent; and after said predetermined time periodinvoking a passive state of the agents, wherein said agents have a freeor paired status and wherein paired agents hold a value determining thequality of the relation they have established.

According to a further aspect of the present invention there is provideda method of operating a computer to implement an agent adapted fornegotiating in a virtual environment, the method comprising: executing afirst program defining an agent descriptor comprising a set ofproperties which determine the negotiating capability of the agent;setting up an agent body by providing at least one sensor for receivinginformation from the virtual environment and at least one actuator forinteracting with the environment; and executing a second program toimplement a decision engine associated with said at least one sensor,said decision engine being operable in an active state during apredetermined time period to execute a decision making process based onthe information received from the at least one sensor according to thenegotiating characteristics of the agent wherein the decision engine isoperable to establish a relation with another agent in dependence on thenegotiating characteristics and to determine a value representing thequality of the established relation.

According to yet a further aspect of the present invention there isprovided A method of operating a computer system to manage demands andresources, the method comprising: generating a scene representing thecurrent state of resources and demands; instantiating a plurality ofagents as executable programs, including a first set of resource agentsrepresenting resources and a second set of demand agents representingdemands, wherein each agent is operable to exchange messages withanother agent and comprises a decision engine for executing a decisionmaking process based on information from the scene and messages fromsaid other agent to implement a relation with said other agent and toassociate a relation value with said relation; and comparing thatrelation value with a reference value for the relation to determinewhether the relation is satisfactory or not.

According to another aspect of the present invention there is provided acomputer system configured as a virtual world representing a real world,the system comprising: a first set of demand agents implemented asexecutable programs, each comprising a set of properties representingdemands in the real world; a second set of resource agents implementedas executable programs, each comprising a set of properties representingresources in the real world; wherein the demand agents and the resourceagents are operable to negotiate by exchanging messages, said messagescontaining said sets of properties corresponding to the demands andresources of each agent so that a decision engine can establish arelation based on predefined criteria; and wherein when a new demand orresource enters the system, an agent representing the new demand orresource is able to disturb an established relation between a pair ofagents by offering compensation to improve a value representing thequality of the established relation.

According to yet another aspect of the present invention there isprovided a computer system comprising a processor for executing aplurality of programs, each program represents an agent for negotiatingin a virtual environment and a memory which stores a plurality of agentdefiners, each agent definer having an ontology comprising: a set ofproperties defining the nature of the agent; a set of attributesdefining characteristics used for negotiating purposes by the agent; aset of scenarios executable by the agent to implement different decisionmaking processes; and a relation quality store for holding a valuedefining the quality of a relation established by the agent.

For a better understanding of the present invention and to show how thesame may be carried into effect, reference will now be made by way ofexample to the accompanying drawings in which:

FIG. 1 shows an example of a virtual world scene;

FIG. 2 shows the logical architecture of a virtual world scene accordingto one embodiment;

FIG. 3 shows a logistic map for an air cargo transportation example;

FIG. 4 shows the negotiations between agents in a first situation of theair cargo example;

FIG. 5 shows the negotiations for self-organisation in a secondsituation of the air cargo example;

FIG. 6 shows an e-commerce application;

FIG. 7 shows the basic agent architecture according to a preferredembodiment of the present invention;

FIG. 8 shows the components of the agent body according to an embodimentof the present invention;

FIG. 9 shows a flow chart of the steps of an agent operational cycle;

FIG. 10 shows an example of an agent's metaontology;

FIG. 11 shows is an example of the ontology of a car agent for thee-commerce application;

FIG. 12 shows a table of a data structure used by the decision-makingmachine according to an embodiment of the present invention;

FIG. 13 shows the steps of negotiations between agents according to anembodiment of the present invention; and

FIG. 14 shows an alternative embodiment of the data structure used fordecision-making machine.

FIG. 15 shows a client-server embodiment for implementing the presentinvention.

Before describing the preferred embodiments of the invention, a briefexplanation of the underlying concept will be given. A virtual world isas the name suggests, an artificial system that is created in an attemptto simulate a real system. The virtual world in the context of thepresent invention is a system where the allocation of resources todemands takes place. Such a virtual world is created by identifying thevarious components of a real world system and then mimicking theirbehaviour using software objects that have certain properties andattributes.

These software objects are called “agents” and are capable ofinterpreting information received from other objects making up thevirtual world. An agent in this type of virtual world may be allocatedto each demand and each resource. Based on inputs, agents can makeknowledge-based decisions, which are then output to other elements ofthe virtual world.

It will be appreciated that the inputs and outputs interface with thereal or physical world to allow a user to control the system and toutilise its results. Interfaces with the real world may take the form ofany known input and output devices for computer systems.

It is possible to create a virtual world using agents having propertiesand attributes which can establish relations with one another. Agents inthe virtual world may be so-called “free” agents where there is norelation established with any another agents. In this case, free agentswill try and establish relations with other agents by matching itselfwith other agents, which it identifies as potential partners usingidentifiable characteristics. Such a matching process is called anegotiation. A relation between agents is only established if all agentsto the negotiation agree and the agreement will be granted only if theproposed relation meets predefined criteria. This criteria might be amatching of certain characteristics between agents, such as a particularproperty, attribute and/or value. This is discussed in more detail inthe following.

A free agent which searches the system for a match will be in an“active” state. If no suitable match is found, the agent switches to a“passive” state and waits for an alteration in the state of the virtualworld. A state of the virtual world at some moment in time is called a“scene”. Agents that have already established relations with otheragents may also switch to a passive state, because they may be activatedagain if there is a change in the state of the virtual world. A changein the “scene” can be caused by a new ‘event’, i.e. the arrival of a newdemand or resource causing the creation of a new agent. Agentscommunicate with each other and with the virtual world by “messages”.

Agents are either “free”, in a “dissatisfied” relation or a “satisfied”relation. The meaning of these terms is discussed later. When there is achange in the state of the virtual world, or at initialisation, theprocess follows the sequence that free agents are activated first,dissatisfied agents are activated second and satisfied agents areactivated last. For purposes of matching, each agent holds a minimum andmaximum value for the property or attribute it is trying to match.Agents seeking to match characteristics can prioritise the particularcharacteristics they require. Agents are said to be ‘satisfied’ if forthe established relation between the agents their respective maximumvalues for characteristics are met. An agent can still establish a‘dissatisfied’ relation if the match is to an agent where the value of arequested characteristic is greater than or equal to the minimum value.A relation will not be established with an agent, if the value of therequested characteristic for that agent is below the minimum value.Thus, a relation may exists between dissatisfied agents, but an agent'sdissatisfaction is a strong reason for reconsidering the establishedrelation and such agents are considered next in the sequence after allfree agents have been checked for a suitable match. Finally, satisfiedagents are activated last to reconsider their links and if it isfeasible the links are reestablished such that the total overall systemvalue is increased.

The maximum value for agents can be set to reflect an overall averagesystem value, for example an average market price, which is mostlyexpected to be achievable. However the average system value can be sethigher than a readily achievable value to see if it can be attained bythe autonomous matching activity of agent pairs, by using compensationto fracture existing relations. In order to break an establishedrelation between agents, compensation is offered to those agents.Compensation will only be accepted if the re-negotiated relation is toresult in a relation, which is more satisfied than the one which wasbroken. The effect of this is to increase an overall system value, whichis a collective value based on the activity of all agents. This overallsystem value may for example be “gross profit margin” (i.e. money) orsome other quantity considered to be paramount to the system. However,it should be appreciated that the overall system value may be gauged interms of “user satisfaction”, or “time”, etc.

It should be noted that a dissatisfied relation could become lessdissatisfied or satisfied. It is also possible of a relation to becomemore satisfied by matching above the maximum value through the offer ofcompensation.

The present embodiment will, as long as time permits or until a newevent enters the system, by using the matching activity of agents havethe effect of continually optimising the overall system value. Thelinear negotiation strategy of starting with trying to match freeagents, proceeding to dissatisfied agents and finally onto satisfiedagents is especially efficient if executed on a powerful executionplatform, wherein agents can keep working to optimise the system.However, because of the dynamic nature of most real systems, it is oftennecessary to limit the time period of agent activity for each new event.

FIG. 1 shows one scene in a virtual world consisting of resource(grey—labelled R) and demand (white—labelled D) agents. There are sevenagents shown 100, 102, 104, 106, 108, 110 and 112 that are interested inestablishing relations. The demand agent 100 has established a relationwith the resource agent 102. This relation is established as indicatedby the solid line 101. The smiling faces indicate that both agents aresatisfied. Two dotted lines 105, 107 extend out from the demand agent106 towards the resource agents 104 and 108 respectively. The dottedlines indicate that these agents are in the process of negotiations, buta valid relation has not yet been established. The smiling facesindicate that the agents are satisfied with the proposed relations sofar. Thirdly, a resource agent 110 is trying to establish a relationwith a demand agent 112. This relation is shown by the dotted line 111indicating that a relation is yet to be established. Furthermore, thesad faces indicate that the agents 110 and 112 do not envisage a goodmatch, but that there is no better relation at present.

Note that either the demand agent or the resource agent can be active.That is, it should be appreciated that for the relation 111, theresource agent 110 initiates the negotiation, which is in contrast tothe other relations 101, 105, 107 of FIG. 1. This is often a preferredstrategy especially in a buyers market, which is in fact the case in thescene of FIG. 1. That is, there are more resource agents (sellers) 102,104, 108, 110 than demand agents (buyers) 100, 106, 112, which meansthere are less buyers to be considered for the matching process.

Once a relation has been established, such a relation can only beaborted by mutual agreement. So in FIG. 1, if a new demand enters thesystem, it might be considered optimal to match a previously assignedresource agent to the new demand agent. However, if this is the case, itwill be necessary for all agents to agree and in order to modify theexisting relation, compensation will need to be offered to the originaldemand agent.

The principle of self-organisation is discussed more fully herein. Fordynamic resource allocation in a self-organising system consider amulti-agent system given the task of allocating n resources to mdemands. In a preferred embodiment, each resource is characterised by aset of f features and having a value expressed in monetary units (mu).Each demand is characterised by a set of g features and havingpurchasing power also expressed in mu. Typically, demands arrive at thesystem one by one where the times of arrival and their characteristicsare unpredictable. Resources may be constant or changeable over time.

A basic example of the algorithm for implementing dynamic resourceallocation, comprises the following steps:

-   1. An agent is allocated to the demand as it arrives to the system.    The allocated demand agent sends its characteristics in a message to    all free agents requesting a resource having particular features    which can be paid for with a particular number of monetary units.-   2. All free resource agents having resources with the requested    characteristics or with at least some of these characteristics make    an offer to the demand agent.-   3. The demand agent then selects the most suitable resource agent    (based on their characteristics) from those offers and a match is    established.-   4. If no suitable resource agent is free, the demand agent seeks to    obtain a suitable resource from a previously allocated resource    agent, by either re-negotiating with the respective dissatisfied    demand agent involved with the established relation or alternatively    offering compensation to a respective satisfied demand agent for    modifying its already established relation.-   5. The demand agent to which the offer of compensation is made then    considers the offer. The demand agent only accepts the offer if the    compensation enables it to obtain a different suitable resource with    improved matching criteria, i.e. closer to (or even above) its    maximum value. If the offer of compensation is accepted, this has    the effect of effectively reorganising the whole system, wherein the    previously established relation between the original demand and    resource agent is modified and a new relationship between the new    demand and the released resource agent is established.-   6. Process steps 1 to 5 are repeated until all resources are linked    to demands and either no established relations can be improved or    until the time period available for allocation has been exhausted.    That is the above process is repeated until an optimal allocation of    resources has been achieved within the constrained time period.

Consider the situation where at one moment in time a scene may existwhere the relations between agents are satisfied and the system is inequilibrium. However at a later time, a new event is input into thesystem. Such an event might be a new order where a demand agent iscreated for this order and the system will autonomously modify existingand/or establish new relations among the agents with the end result ofincreasing the overall system value. This capability of autonomouslyre-negotiating relations in a multi-agent system is known as“self-organisation”. A multi-agent system has a system in equilibrium atone moment in time where resources and demands are matched optimally.Then if an event such as a new demand disturbs the system, the systemdynamically allocates resources by performing self-organisation and at alater time the system is again in equilibrium, although now therelations may have been modified with the effect of optimising theoverall system value by offering compensation to agents with existingrelations.

Dynamic allocation of resources is where resources need to be allocatedor demands for these resources change during the allocation process inan unpredictable manner. Examples of dynamic resource allocationinclude:

-   -   Production logistics, which relates to the allocation of        materials or components to production facilities, for example        assembly robots, conveyors, metal processing machines, etc. A        logistics application may have the added complexity of requiring        components to be specified in time and space (see later) or        frequent changes might be made to product specifications.    -   E-commerce, which relates to the allocation of goods or services        (supply) to customer requests (demand) when customers or        suppliers unpredictably join or leave the allocation process.    -   Staff scheduling, which relates to the allocation of tasks to        staff in a business operating in a highly dynamic market.

The first two applications will now be described in more detail.

FIG. 2 shows the logical architecture of a virtual world scene accordingto one embodiment where every virtual world scene is defined in terms ofdoubly linked lists. Such a logical architecture is used by a processorto execute the various elements of such a scene. Consider a first scene250 defined by the lists 252 and 254 where the first panel list 252 is alist of 1 to N descriptors of agents used in the scene and the secondpanel list 254 is a list of 1 to M relations between agents in thescene. The relations 264, 268, 270 also contain a status of the list ofrelations. So FIG. 2 shows that there is a first relation 264 between afirst agent 258 and a second agent 260 and it is an establishedconnection, whereas a second relation 268 is between a plurality ofother agents but that the status is still in the booking stage wherenegotiation are on-going between suitable agents but an establishedrelation is yet to be reached.

As a consequence of these doubly linked lists 252, 254 a processor isable to proceed from every agent 258, 260, 262 or relation 264, 268, 270to its neighbour. The logical architecture of this embodiment allowsthese lists to refer to one another so that agents can find theirrelations and also other agents, with whom these relations have beenestablished. In this embodiment, the agent and relation panels 252, 254are also divided into panels of active and inactive and satisfied anddissatisfied agents in order to reduce the processor time for searchingthe execution queues to the system dispatcher. The system dispatcher isresponsible for generating the clock cycle for agent operations and forcontrolling the execution of the objects of a virtual world scene.

Consider the specific logistics application of an airship cargotransportation system as shown in FIG. 3. For logistics applications itis often necessary to consider additional factors such as space andtime, which requires a scaled map of site locations and routes withtheir respective dimensions and to consider at what time a resourceshould be allocated to a demand. FIG. 3 shows the logistics map of theairship cargo virtual world where there exists a network of sites 120,122, 124, 126, 128, 130, 132 for loading and unloading cargo fromairships, each site being labelled with a capital letter A to F and thecentre site with letter O. Also, the routes between these sites areshown with their respective dimensions d1, d2 and d3 where d1=500 km,d2=250 km and d3=150 km. Two airships are used to transport cargobetween the sites. Agents are assigned to each of these airships and arereferred to herein respectively as “A1” and “A2”. FIG. 4 shows thenegotiations that occur between a new order assigned the agent referredto herein as “O1” and the agents A1 and A2.

The airship represented by agent A1 has a cargo capacity of 100 tons,moves at 10 km/h and has relatively high operating costs of 7 mu/km. Theairship represented by the agent A2 has a cargo capacity of 10 tonsmoves at 15 km/h and has an operating cost of 3 mu/km.

Situation 1:

On 15 September the system receives an order for the transportation of a5-ton cargo from site A to site D (500 km distance) for 15000 mu. Thedeadline for the cargo to reach site D is 20 September. A1 is free andlocated at site E, while A2 is currently transporting a previouslyaccepted order to site C. To plan the operation, a first order agent O1is created and assigned to the order. Agent O1 sends a request withdetails of the required cargo transportation to both A1 and A2. Havingconsidered the request, each airship sends its offer to O1.

A1 is ready to start the transportation immediately. To accept theorder, the airship needs to move from site E to site A (300 km). Thecost is (300+500)*7=5600 mu and the time required for the execution is(300+500)/10=80 hours, ie, about 3.5 days. Therefore, A1 can meet thedeadline of 20 September. A2 will have to complete its currenttransportation task first and then move to site A to undertake theexecution of the new order O1. If A2 is currently located at site O, itwill need to cover the route O-C-O-A (750 km) and then move from A-D(550 km) to complete the order A1. The time required for the executionwill be (250+250+250+500)/15=83,3 hours, ie, also about 3.5 days andthus also will meet the deadline. The cost will be(250+250+250+500)*3=4500 mu.

Both A1 and A2 can handle a 5-ton cargo and both agents can meet thedeadline of 20 September. Therefore the criteria to consider is which isthe more costly agent. The result is that A2 is cheaper at 4500 mu,which results in a greater profit margin of 15,000−4500=11.500, whereasfor A1 the profit margin is only 15,000−5600=10,400 mu.

FIG. 4 shows the negotiations between agents where the line 141 denotesthe enquiry sent by O1 to A1. Line 146 denotes the reply (offer) fromA1. Line 148 denotes the enquiry sent to A2. Line 150 denotes the reply(offer) from A2. Line 152 denotes the booking of A2 and the establishedrelation between the agents O1 and A2.

Situation 2:

Now consider the case where one hour later, a new order for cargo isreceived by the system and is assigned a new order agent refereed toherein as “O2”. That is, an 8-ton cargo needs to be transported frompoint C to point B (150 km) for 20,000 mu. The deadline is 17 September.Only A2 can execute this order, because A1 would not be able to meet thedeadline since A1 would need (500+150)/10=65 hours, or about 3 days andtherefore must not be able to meet the required deadline of 17 September(2 days away). In contrast, A2 would need only (250+150) 15=23,3 hours,or about 1 day to complete this order. However, A2 has already beenbooked by O1 in situation 1 above.

Thus, an enquiry from the agent O2 to agent A2 results in the followingnegotiation:

-   -   A2 sends a message to O1 requesting permission to be released        from the previously established relation with a view to        accepting O2 and offering in return an unspecified compensation    -   O1 attempts to oblige, by contacting A1 and A2 again and        receives from A1 the same offer as in situation 1, which meets        its deadline, but was previously rejected as the more expensive        option (10400 vs 11500 mu)    -   O1 asks A2 to compensate the difference, ie. 1100 mu    -   The O2 agent considers the request for compensation passed to it        by A2. If O2 establishes a relation with A2, O2's profit will be        20 000−(250+150)*3=18950 mu. Therefore, the compensation to be        paid 1100 mu to the O1 is acceptable in light of the profit        which A2 expects to make from the deal 18950 mu.

FIG. 5 shows the negotiations for self-organisation of the system toarrive at an improved overall result for the system. Line 154 denotesthe enquiry from O2 to A2. Line 156 denotes the proposal for re-bookingfrom the A2 to O1. Line 158 denotes the enquiry from O1 to A1. Line 160denotes the positive response from A1 to O1. Line 162 denotes thepermission given by O1 for A2 to be re-booked. Line 164 denotes thecompensation of the O1 offer, sent by A2 to O2.

In summary, A2 is re-booked and A1 now established a relation with O1,which allows A2 to establish a relation with O2 which improves theoverall value of the system

A second application involving electronic commerce (E-commerce) will nowbe described.

Consider the example of selling cars over an Internet portal using aninternational network of dealers in Germany and Brazil having their ownwarehouses and transportation means for the delivery of cars. An agentis assigned to each order, dealer, warehouse, transportation unit,transportation slot, transportation load and car. These agents arecapable of not only selling cars as is the case in existing e-commercesystems, but are also capable of self-organisation in a dynamicallychanging environment, thereby increasing the overall system value whilestill meeting the requirements of all customers. As will be described inmore detail later, the agents are imbued with sufficient intelligence sothat they can act autonomously by carrying out the cancellation ofestablished relations (i.e. contracts) or the modification of theserelations with the buyers' and sellers' permission.

FIG. 6 shows an e-commerce application. A first order 212 is from afirst customer residing in Brazil for a red Mercedes cabriolet 202 witha price of 50000 mu. The car has already been paid for and is now 300meters from the customer's house, being delivered by a truck from theBrazilian dealer 214. The first customer has been waiting for it for onemonth already.

A second customer in Germany urgently wants to have exactly the same carand places a second order 210. Dealers in Germany 204, 206, 208 cannotfind any models_of this car that match the second customers requirementsin their warehouses 204, 206, 208. The multi-agent system will now tryto generate for the first customer an alternative offer, which would bedifficult to reject, with a view to delivering the red Mercedes 202 tothe second customer.

To achieve this, the second order agent 210 sends a message to the Caragent 202 (i.e. red Mercedes) with an offer, where the compensation isnot specified, to induce it to change the customer. The Car agent 202,in turn, sends a message to the first order agent 212 asking forpermission to change the customer. In response, the first order agent212 considers the situation taking into account its own and the carowner's interests. First, the first order agent 212 turns to theBrazilian dealer 214 to find out how fast it would be possible todeliver a new car with the same characteristics. Assuming a new car ispossible to be delivered within one month, this doubles the waitingperiod of the first customer. In accordance with a simple scenariostrategy (see later) of proportional compensation, the Order 1 Agentasks the second order agent 210 for a 50% discount as compensation forthe doubling of the delivery period of the car. To obtain permission forthe deal, the first order agent 212 contacts the first customer, forexample, by ringing his WAP phone, and offers a 50% discount in exchangefor the agreement to wait another month. If the first customer agrees towait another month, the first order agent 212 cancels the booking of theinitial red car 202 and books a new car 220. The second order agent 210in turn books the car 202 for the price which now equals its initialprice plus the amount of compensation: 50,000+25,000=75,000 mu.

Now, the second order agent 210 has to negotiate with an airliner forthe delivery of the car back to Germany within 2 weeks. Assuming thatthe airliner agrees, where even if all the flights have been fullybooked negotiations can be involved with other agents representing bigcargo loads to give up their transportation slots in return forcompensation. Thus assuming the compensation for air transport amountsto 4000 mu, the total cost of the car is now: 50,000+25,000+4,000=79,000mu.

This price is offered to the second Customer in Germany, who willreceive the negotiation report and see the decision tree (with theurgency of his order taken into account) if required. The secondcustomer can then either accept or reject this offer. If the customeraccepts, the truck driver delivering the car to the first customershouse will re-route to the airport and load the car onto the designatedflight to Germany. If the customer rejects, all the achieved relationsare cancelled and the initially established relation is reverted to,i.e. the car is delivered to the first customers house as planned.

If compensation is accepted and both orders are executed successfully,the agents gain additional profit by meeting the demands of eachparticipant of the deal (ie. Customer 1, Customer 2, airliner, etc.).Therefore through a process of self-organisation, the present inventionis able to achieve dynamic changes with regards to relations between themarket participants. It should be understood that to gain the fullbenefits of an e-commerce application, ideally there should be no fixedprices and no fixed terms of delivery, instead everything should dependon customers' requirements.

To achieve the possibility of carrying out self organisation asexemplified and discussed above, a particular agent architecture isdescribed. FIG. 7 shows the basic agent architecture, which has two mainparts; an agent descriptor 4 and an agent body 6. The agent descriptor 4contains a set of properties 8, 10, 12 of the agent 2 or itsrelationship with other agents. The properties of an agent determine thenature of the agent. An agent property is able to refer to differentscenarios 14. Each scenario can be implemented as a pre-programmedprocedure that resides in a command memory 14 portion of an agent (seeFIG. 8). An agent can select whichever scenario it feels will producethe optimum result. The scenario provides information on the manner ofestablishing a desired relation with another agent or can calculatevalues to check the suitability of a property for matching purposes.

Each property 10 is also capable of referring to a set of attributes 16for matching purposes. For example, the car agent 202 in the e-commerceapplication has the property “colour” and the attribute “red” assignedto this property. An agent 2 has certain parts of it that can be read,for example, the agent descriptor 4 can be made “open”. In the exampleshown in FIG. 7, property 8 is made open so that other agents can readthis property provided they have a so-called virtual “visual sensor”.Other parts of the agent descriptor 4, for example, properties 10 and12, may not be opened and can only be opened by agents on request.

A visual sensor is the mechanism that is used by agents to read the opendata fields of an agent descriptor 4. Typically, the sensing mechanismconsists of a software procedure and data structure built into the agentbody or alternatively the vision sensor mechanism can be transferred toan agent upon request from a base class held elsewhere in the system.The vision sensor mechanism can also have filters, which impose visionlimitations such that only certain open parts of an agent descriptor canbe read.

FIG. 8 shows the components of the agent body 6. The agent body hassensors 20, actuators 22, scene memory 24, a decision-making machine 26,ontology memory 28, fact memory 30 and command memory 14. Examples ofsensor elements 20 include timers, vision sensors, mailbox mechanisms,etc. Some of these interact with other agents and some with the realworld. The actuators 22 may include WAP-phones, email, and means ofaccessing an agent's database, etc. Thus, these allow establishedrelations to be implemented in the real physical world. Ontology memory28 is the knowledge-base that an agent possesses. Although the ontologywill be discussed in more detail later it can be broadly thought of asbeing a knowledge-base that can be split into two categories being a“general” knowledge base or a “domain-specific” knowledge base. Althoughmost agents can operate fairly well with a general knowledge base it hasbeen found more efficient in certain applications to use moredomain-specific knowledge bases for example in a car productionapplication. The sensors 20 can be used to build-up a scene memory 24 sothat the agent has an idea of the surrounding virtual world. Thedecision-making machine 26 is the core of the agent body and interfaceswith most of the other elements. It uses its own knowledge to make adecision based on its received inputs and to implement the requiredcourse of action to be output. In reaching a decision thedecision-making machine 26 can also select a preferred action scenario14 for a particular situation. For certain applications one might needthe scene memory 24 but for most simple applications (for example emailapplications) agents really just need a general ontology memory 28 ofthe network.

The operation of each agent is based on a clock cycle which for examplein the present embodiment is 300 μs. This clock cycle is allotted to allagents by a system dispatcher (not shown). The system dispatcher can beconsidered to be a unique and independent software object, which isexecuted on the processor to organise the activities of the agents. Thesystem dispatcher is able to provide certain information to other agentsin the virtual world. For example if an agent would like to have avirtual vision sensor if can make such a request to the systemdispatcher which then forwards the relevant software to the requestingagent.

FIG. 9 shows a flow chart describing the steps of an agent operationalcycle. Step S40 indicates the start of the agent operational cycle whenan agent receives a clock edge from the system dispatcher. All of theactive agents in a scene operate in synchronisation by working inparallel for each clock cycle. At step S44 each agent decides whetherany important events have occurred in the scene. This decision isaffected by the input sensors where in FIG. 9 a simple agent having amailbox input sensor S42 is shown. Each agent performs a check onwhether the clock cycle has run out from step S60 and receives anyinputs from the mailbox at S42. If there is a new event in the scenethen at step S46 the current scenario is delayed and at step S48 a newscenario is selected based on the agent ontology 28. That is, the agentmight decide based on the ontology to execute a different scenario,which will be most better suited, to handle the new event. If there areno new events in the scene at step S44, the agent then proceeds to stepS54 where it chooses the next command of the current scenario. Also atthe end of step S50 once the new scenario has been executed the agentmoves to step S54. At step S56, the next command of the current scenariois executed and output to an email actuator mechanism 22. At step S60the agent checks whether the clock cycle has expired. If not, then thealgorithm is returned to step S44 where the agent checks whether anyother important events have entered the scene. If the clock has expired,step S62 stipulates that the end of the agent operational cycle has beenreached.

As explained, the embodiment of the agent of FIG. 9 has a mailbox as theonly input sensor and email as the only actuator. For such a simpleagent it is not required to have a scene memory 24, instead a generalagent ontology 28 is probably sufficient. However, it should beappreciated for more complex applications, for example the logisticsexample described earlier, a scene memory 24 would be required inaddition to the ontology memory 28. If a scene memory is not used suchan agent would not know how many other agents there exist in the virtualworld and what kind of agents these are.

The virtual decision-making machine 26 is the core element of the agentbody and interfaces either directly or indirectly with the otherelements. The decision-making machine 26 allows each agent to select aparticular scenario from a set of these scenarios by comparing theirdescriptors. A scenario descriptor is a shortened piece of software codeproviding an overview of the full scenario. The decision making machine26 takes into account the current situation of the system using theontology and/or scene memory where a particular scenario will ultimatelybe selected which most efficienty meets the required criteria. Theselected scenario is then activated and executed as a plurality ofcommands in real time. So as an example, one scenario might be amatching algorithm or another scenario might be to offer compensation,etc.

The ontology memory 28 is a knowledge base where knowledge is stored interms of:

-   1. objects-   2. relations-   3. properties-   4. scenarios, and-   5. attributes.

Metaontology is a description of all knowledge bases and describes theoverall structure of ontologies. FIG. 10 shows how the basic elements ofan agent ontology are interconnected. More specifically, metaontology isa semantic network having basic elements shown in the rectangular boxesof FIG. 10 as nodes and the various relations between them indicated byarrows together with the actual scenarios 14 for their interpretation.These elements and relations contain all the required knowledge forrunning a multi-agent system. Using metaontology, any domain knowledgecan be constructed as a network of elements and scenarios. For examplein the logistic and e-commerce applications described earlier theontology will include the specification of ordered products, types ofproducts, time needed for their production, necessary components,conditions and costs of storing components in warehouses, etc.

Certain parts of the ontology memory contain knowledge which is commonto all the agents. For example, the ontology containing the catalogue ofcomponents, including production times and costs is common knowledge.This common knowledge may reside in an element common to the system suchas the system dispatcher or alternatively this knowledge could be placedin the ontology memory 28 of each agent. Since each agent contains aprivate ontology memory it is able to provide independent access to thismemory and avoids conflicts from agents trying to access this memory.

As an example consider the e-commerce application where the generalisedontology of a car agent is shown in FIG. 11. A car agent 70 is shown ashaving three properties: the model 72, the price 76 and the colour 74.The car 70 also consists of the relations 78 and 80. The relation 78consists of the properties of a car 70 which are the body 82, the engine84, and the car frame 86. The car also has a relation 80 consisting ofthe model range which may be one of the following properties: a luxurymodel 88, a standard model 90 and a sport model 92. The price property76 is shown to include a set of discount calculation action scenarios14. An agent is then able to select the required scenario depending onthe discount calculation required. The price property 76 includes aprice attribute 94, which for example gives the price range that wouldbe acceptable to a potential buyer, for example ranging from US$10,000to US$25,000.

It should be appreciated that there are various different discountscenarios 14 that an agent can select for the ontology shown in FIG. 11.For example, in the e-commerce application a proportional discountstrategy was selected where a discount of half the car price was offeredas compensation for doubling the delivery time of the car. However, thefollowing discount scenarios might also for example be available: a“regular customer” discount depending on the number of the previous carpurchases made; a “wholesales” discount depending on the number of carspurchased with one order; a “purchased with delayed delivery” discountwhere the customer pays in advance but can accept a delay in thedelivery time or a “competitors prices” discount where the car price iscomparable to or lower than competitors prices.

The action scenarios 14 may also include decision-making rules which arestored in the respective scenarios. For example, a scenario may containrules of resource allocation based on profitability estimation or rulesof estimation criteria defined by users.

The virtual decision-making machine 26 is a core element of the agentarchitecture (see FIG. 7). The machine 26 is implemented in a preferredembodiment as a set of procedures and data structures. The proceduresare written for handling the basic data structures and actions requiredof an agent. For example, organising information exchange among agents,collecting information about supply and demand dynamics, findingpossible connections between agents, selecting the best connection,executing decisions and scenarios or reconsidering them if the situationchanges.

As explained before each agent is described by a list of its propertiesand their attributes. Each agent can have properties that haveattributes where each attribute may have values. Alternatively,properties can also have values. So in the example of FIG. 11, the CARagent 70 has a PRICE property 76, which has an attribute PRICE RANGE 94from $10,000 to $25,000 where for a particular car the average car valuemight be $15000. So an agent may have a data structure as follows:<Property 1=Attribute 1>, <Property 2=Value 1>, . . . etc. To learnabout agent properties, attributes and values, agents can exchangemessages such as one agent asking another whether it has property 1 orwhether the other agent has attribute 1 assigned to property 1, etc.

Some properties can be made “pen” so that these properties are visibleto other agents. This reduces the number of request messages and thecomplexity of the scenario logic needed. Decision making is performed onthe basis of the information collected from messages, by ‘seeing’ or‘reasoning’ agent properties and attributes and from agent ontologies.Agent ontologies typically contain knowledge of the overall virtualworld model and with the scenarios govern agent behaviour in differentsituations.

The typical data structure for decision-making is a table of offers withinformation about received proposals. An example of such a table for ademand agent in a logistics application is shown in FIG. 12. In thissimple example, only three properties are shown: product class, costsand time of delivery.

The structure of the table of FIG. 12 will vary, depending on the numberof properties that have been defined for an agent. The virtualdecision-making machine can perform the following basic tableoperations: creating/deleting the table, defining or clearing fields,sorting offers according to a field and searching for a string usingpatterns.

The most important step is to choose the best offer. This procedure canbe very complex. The simplest case is when there is a full matchingpossible between demand and supply so in the present embodiment whenboth the resource agent contains all the properties requested by thedemand agent, i.e. the class of product, price and time of delivery areall matched. A more complex case is when demand and supply are close toeach other as measured by an agreed metrics, i.e. partial matching.Partial (dissatisfied) matching may be when one of the properties ismatched, for example if the resource agent has the same product classbut the cost and time of delivery cannot be satisfied.

In a preferred embodiment, to accomplish matching between demand andresource properties a demand agent sends a request, indicating therequired criteria for matching of an agents properties and their values.For example, the request message might be ‘Are there any A classresources with value Z1 for property A1, importance B1, and with valueZ2 for property A2, importance B2?’ etc. This query may be eitherdirected to a group of agents or broadcast to all agents. Resourceagents then check their corresponding resource properties and theirattributes in the ontology memory and respond. Even if they do rot matchthose indicated in the request, resources can be offered as possiblecandidates for partial matching.

There are two ways to perform the priority-based demand-resourceproperties matching:

-   1. Sequential matching of properties, starting with top-priority    properties. The first demand agent sends a message to resource    agents: “I am class N Demand Agent. If there are any class N    Resource Agents with name S1, specify your value”. After it receives    their replies, the demand agent creates a ‘property—value’ data    structure table. From the table it chooses one or more suitable    offers and sends to their agents the question: “Do you have property    S2? If yes, then what value does it have?” This way of performing    matching allows the agent to: proceed from more important properties    to less important; apply the partial matching strategy (i.e. by    possibly not paying attention to properties of minor importance);    reduce the number of comparing operations; keep its closed property    values hidden and changing the accessibility in accordance with its    negotiation strategy or market situation (for example if the number    of competitors changes).-   2. Parallel matching through a mediator. Demand agents disclose    their open and closed properties to a mediator, who calculates    indexes and returns them to agents. Agents do not know exact values    of other agents' properties, but are aware of the degree of their    similarity or difference as indicated by indexes.

FIG. 13 shows an embodiment of the steps of negotiations between agents.At step S300 a scenario is run for establishing relations then at stepS302 a demand agent reads from its ontology memory 28 a list of suitableresource agents. The demand agent then sends request messages S308 toall suitable resource agents at step 306. Step S306 may also beperformed if there are any changes in the virtual world scene S304. Atstep S310 the demand agent waits for a replies from the resource agentsS312. Then, at step S314 a decision making table as shown in FIG. 12 iscompiled. At step S316, the best offer can be selected where at stepS318, a request message is sent for establishing a relation with thechosen resource agent S320. At step S322, the demand agent waits for thereply message from the resource agent S324. If the reply is negativeS326 then the negotiation procedure loops back to step S318 where thedemand agent makes a proposal to the next best resource agent. If theanswer is positive S328, then a relation is said to be established.Finally at step S300, once a relation has been established the demandagent switches to its passive mode until such time as there are changesin the scene S304.

A further embodiment of a data structure that is used by thedecision-making machine is shown in FIG. 14. FIG. 14 gives the agent acurrent indication of the market, where for each product the currentaverage price is calculated based on a history of negotiations betweenagents and the current demand for the product in the market. The agentis then able to determine whether there is currently an increase ordecrease in the average price of the product. For example, if there is adecrease in the market it might it might be better for a resource agentto wait until the market recovers.

It should be appreciated that a multi-agent system according to thepresent invention might be implemented using the client-server basedembodiment shown in FIG. 15. More specifically, processing and executionof the agents is performed by a server, whereas client terminals operateas a user interface allowing users located throughout the world to inputor output information to or from the system. The human user can thusenter information, or receive information from the system where forexample, a car salesman through his experience will have knowledge as tothe average market price of a car having certain characteristics. Thecar salesman is then able to enter this information into the system. Theclient terminal transmits the entered information to the server forprocessing such that the ontology of each of the relevant agents in thesystem is updated with this knowledge, which may be used for matchingpurposes.

1. An agent adapted for negotiating in a virtual environment, the agentcomprising: an agent descriptor implemented as an executable program andcomprising a set of properties which determine the negotiatingcharacteristics of the agent; an agent body implemented as an executableprogram and comprising at least one sensor for receiving informationfrom the virtual environment, at least one actuator for interacting withthe environment, and a decision engine operable in an active stateduring a predetermined time period to execute a decision making processbased on the information received from the at least one sensor whereinthe decision engine is operable to establish a relation with anotheragent in dependence on the negotiating characteristics and to determinea value representing the quality of the established relation; andontology memory for storing knowledge concerned with the characteristicsof and relations between agents in a multi-agent system, wherein anagent is able to refer to its ontology memory to find a list of suitableagents with which to negotiate.
 2. An agent according to claim 1,wherein at least some of said properties are associated with a set ofattributes which define the nature of the agent.
 3. An agent accordingto claim 1, wherein at least some of said properties each refer to a setof scenarios, each scenario comprising a sequence of commands thatgovern the behaviour of execution of the agent.
 4. An agent according toclaim 3, wherein said decision engine is operable to select from saidset of scenarios the scenario considered to be most appropriate based onthe information received from the at least one sensor according to thenegotiating characteristics of the agent and the quality of any existingestablished relation that the agent has made.
 5. An agent according toclaim 3 wherein one of said scenarios is configured to cause thedecision engine to disturb an established relation between a pair ofagents by offering compensation which will improve said value.
 6. Anagent according to claim 3, wherein the agent body comprises commandmemory for holding said scenarios.
 7. An agent according to claim 1,wherein the virtual environment represents a real environment includingdemands and resources, and wherein said agent is a demand agentallocated to a real demand.
 8. An agent according to claim 1, whereinthe virtual environment represents a real environment including demandsand resources, and wherein said agent is a resource agent allocated to areal resource.
 9. An agent according to claim 1, wherein the agent bodycomprises a scene memory holding data representing the current state ofthe virtual environment.
 10. An agent according to claim 1, wherein theat least one sensor comprises means for reading accessible databasefields.
 11. An agent according to claim 1, wherein the at least onesensor comprises a mailbox mechanism for receiving messages.
 12. Anagent according to claim 1, wherein the at least one actuator comprisesmeans for accessing a database to update data fields therein.
 13. Anagent according to claim 1, wherein the actuator comprises means fordispatching a message.
 14. An agent according to claim 1, wherein theagent body comprises a timer for controlling said predetermined timeperiod.
 15. An agent according to claim 1, wherein said ontology memorycontains common knowledge which is accessible by all agents in amulti-agent system and private knowledge which can only be accessed bysaid agent.
 16. An agent according to claim 1, wherein saidcharacteristics of an agent comprise properties, attributes and values.17. An agent according to claim 1, wherein said ontology memory isdomain specific so that an agent may be specially imbued with theknowledge of a particular application.
 18. An agent according to claim17, wherein said application is a logistics application.
 19. A method ofsimulating a real environment by creating a virtual environmentcomprising a plurality of agents, the method comprising: for at leastsome of said agents, invoking an active state of the agent for apredetermined time period; for each active agent, implementing adecision making process based on the information received from the atleast one sensor according to the negotiating characteristics of theagent to determine whether a relation can be established with anotheragent; after said predetermined time period invoking a passive state ofthe agents, wherein said agents have a free or paired status and whereinpaired agents hold a value determining the quality of the relation theyhave established, wherein each agent comprises an agent descriptorimplemented as an executable program and comprising a set of propertieswhich determine the negotiating characteristics of the agent; an agentbody implemented as an executable program and comprising at least onesensor for receiving information from the virtual environment, at leastone actuator for interacting with the environment, and a decision engineoperable in an active state during a predetermined time period toexecute a decision making process based on the information received fromthe at least one sensor wherein the decision engine is operable toestablish a relation with another agent in dependence on the negotiatingcharacteristics and to determine a value representing the quality of theestablished relation; and ontology memory for storing knowledgeconcerned with the characteristics of and relations between agents in amulti-agent system, wherein an agent is able to refer to its ontologymemory to find a list of suitable agents with which to negotiate.
 20. Amethod of operating a computer to implement an agent adapted fornegotiating in a virtual environment, the method comprising: executing afirst program defining an agent descriptor comprising a set ofproperties which determine the negotiating capability of the agent;setting up an agent body by providing at least one sensor for receivinginformation from the virtual environment and at least one actuator forinteracting with the environment; executing a second program toimplement a decision engine associated with said at least one sensor,said decision engine being operable in an active state during apredetermined time period to execute a decision making process based onthe information received from the at least one sensor according to thenegotiating characteristics of the agent wherein the decision engine isoperable to establish a relation with another agent in dependence on thenegotiating characteristics and to determine a value representing thequality of the established relation; and loading into a command memory aset of scenarios to which at least some of said properties refer, eachscenario comprising a different set of commands.
 21. A method accordingto claim 20, wherein said decision making process selects the mostappropriate scenario from a set.
 22. A method according to claim 20,which comprises compiling in a scene memory a current scene of thevirtual environment for access by the agent.
 23. A method according toclaim 22, wherein the current scene is altered on the arrival of a newevent and if the event is important then the decision making processselects a different scenario.
 24. A method according to claim 23,wherein said new event is a new agent.
 25. A method according to claim23, wherein said new event is a message from an existing agent in thescene.
 26. A method according to claim 23, wherein if a new event entersthe scene but the event is not important then the decision makingprocess continues to execute the current scenario.
 27. A method ofoperating a computer system to manage demands and resources, the methodcomprising: generating a scene representing the current state ofresources and demands; instantiating a plurality of agents as executableprograms, including a first set of resource agents representingresources and a second set of demand agents representing demands,wherein each agent is operable to exchange messages with another agentand comprises a decision engine for executing a decision making processbased on information from the scene and messages from said other agentto implement a relation with said other agent and to associate arelation value with said relation; and comparing that relation valuewith a reference value for the relation to determine whether therelation is satisfactory or not; and loading into a command memory a setof scenarios to which at least some of said properties refer, eachscenario comprising a different set of commands.
 28. A computer systemconfigured as a virtual world representing a real world, the systemcomprising: a first set of demand agents implemented as executableprograms, each comprising a set of properties representing demands inthe real world; a second set of resource agents implemented asexecutable programs, each comprising a set of properties representingresources in the real world; wherein the demand agents and the resourceagents are operable to negotiate by exchanging messages, said messagescontaining said sets of properties corresponding to the demands andresources of each agent so that a decision engine can establish arelation based on predefined criteria; wherein when a new demand orresource enters the system, an agent representing the new demand orresource is able to disturb an established relation between a pair ofagents by offering compensation to improve a value representing thequality of the established relation; and wherein each agent has ontologymemory for storing knowledge concerned with the characteristics of andrelations between agents in a multi-agent system, wherein an agent isable to refer to its ontology memory to find a list of suitable agentswith which to negotiate.
 29. A computer system comprising a processorfor executing a plurality of programs, each program represents an agentfor negotiating in a virtual environment and a memory which stores aplurality of agent definers, each agent definer having an ontologycomprising: a set of properties defining the nature of the agent; a setof attributes defining characteristics used for negotiating purposes bythe agent; a set of scenarios executable by the agent to implementdifferent decision making processes; and a relation quality store forholding a value defining the quality of a relation established by theagent, wherein said ontology memory contains common knowledge which isaccessible by all agents in a multi-agent system and private knowledgewhich can only be accessed by said agent.